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In this paper the National Assessment of 
citizenship's objectives, procedures, exercises, and findings are 
considered as a source of implications for further research in the 
area of social studies. First, the methodology of National Assessment 
can be used as a model for developing objectives, and for devising 
exercises to measure attainment of social studies objectives. Social 
studies curricula now depend highly upcHi standardized tests which 
compare each student with the average performance of all students, 
emphasize the average range, and require each respondent to answer 
all items. In contrast. National Assessment instruments are 
critericn-iTeferenced measuring the performance of groups of 
students, are geared to the ability of the high, average, and low 
student, and employ a sampling matrix. Secondly, the materials and 
exercises are available to researchers and have application to other 
areas in the social studies. Lastly, National Assessment data at 
their disposal, social studies researchers can interpret, apply and 
extend the findings in numerous ways. (SJM) 
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Assessing citizenship attainments may well be one of the most challenging 
measurement tasks facing educators and educational researchers. When I was 
in elementary school my teachers did not realize this. My citizenship period- 
ically received a global rating of "satisfactory" or "unsatisfactory," I 
imagine my teachers found this two-point rating scale very easy to use, and 
in a self-contained classroom there was no need to be concerned about estab- 
lishing interrater reliability. Citizenship in this sense of "deportment" is 
still rated on report cards in many schools, but with the added methodological 
sophistication of five-point scales and multiple ratings. 

My high school government teacher believed that good citizens knew everything 
in Magruder's, so that in one six-week summer course I learned tfie difference 
between robbery and burglary, and several other important facts which I no 
longer remember. Since there were questions at the end of each chapter, with 
answers on the teacher's manual, she didn*t have any evaluation problems either. 

The efforts of the National Assessment of Educational Progress to obtain measures 
of the important educational achievements of Americans have gone far beyond 
these earlier perspectives on the meaning of good citizenship. The National 
Assessment of Educational Progress was designed to provide census-like data 
about what important things students and young adults know and can do in ten 
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subject-matter areas: Art, Career and Occupational Development, Literature, 
Reading, Writing, Music, Science, Mathematics, Social Studies, and Citizenship 
In 1969-70 the first actual assessment was made— in the areas of Citizenship, 
Science, and Writing. The citizenship results reported in this paper are 
from this 1969-70 data collection. Since the ten areas are assessed in cycles 
citizenship is projected to be assessed again in 1974-75. 

The task of assessment of citizenship demanded careful specification of wJ"at 
knowledge and behavior was considered desirable citizenship in the form of 
statements of objectives. Existing lists of objectives were scanned, and 
teachers and students were asked to describe incidents which they felt 
revealed "good" and "bad" citizenship. Teams of reviewers, consisting of 
educators 5 political scientists, persons active in public life, and concerned 
lay people from all over the country carefully examined and modified the 
objectives until there was a high degree of consensus that the resulting 
objectives were indeed the most important objectives to measure. Every 
objective assessed had to meet the following criteria: 1) be considered 
important by scholars, 2) be accepted as an educational task by the schools, 
and 3) be deemed desirable by leading lay citizens. The most current version 
of the citizenship objectives is summarized in Appendix A. 

Measures of these objectives were then developed, reviewed by panels of 
reviewers, and revised. While the format of many of the exercises is quite 
familiar— multiple-choice questions asking for specific knowledge, and short- 
answer qut'Stions--other situational exercises assessed ability to cooperate 
effectively in a group task and willingness to express views on controversial 
issues publicly. 



This direct link between each objective and the exercise(s) measuring it allows 
National Assessment to report results separately for each objective. Results 
in each subject area are reported by; 

Four age levels--9, 13, 17 and young a:lu' : (26-3r) 
Seven types of community (extreme inner city, inner city fringe, 
extreme affluent suburb, suburban fringe, medium sized city, 
small city and extreme rural) 
Four geographical regions (Northeast, Southeast, Central, West) 
Four educational levels of parents (not more than 8th grade, more 
than 8th grade but less than high school gradiation, high school 
graduation, some formal education btyond high school) 
Race (Black, Other) 
Sex 

For example, under the general objective of showing concern for the well-being 
of others, several items tapped respondents' concern about racial equality. 
Two of these exercises are reproduced in the Appendix, along with national 
achievement levels. To the first, a hypothetical case of minority group 
children being excluded from a park, the great Majority c ' respondents [19% 
to 90%) at three age levels (13, 17, and adult) said they should do something 
about the situation. At least 80% of each group named one or more acceptable 
ways to help. 

Thirteen- and seventeen-year olds were asked how willing they would be to asso- 
ciate with a person of a different race in five situations. While close to 90'^, 
at each age level were willing to do so in at least three of the situations, 
57% of each age group expressed willingness to associate in all five situations. 
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The results of administration of the citizenship assessment should be of 
considerable interest to a wide audience--] ay people who want to see v/hat the 
schools are accomplishing; educators in search of objectives, criterion- 
referenced measures, and guidance in curriculum improvement; and to social 
studies researchers. Typically the findings of a research project are considt.^d 
the source of implications for further research. I have taken the time to give 
an overview of National Assessment procedures and of the objectives and exer- 
cises used in the assessment of citizenship, because the methods and materials 
of the citizenship assessment— as well as the data from exercise administration- 
can be used by social studies researchers to explore several interesting and 
important areas. 

First, the methodol ogy of the National Assessment can be used as a model for 
developing consensual objectives and for devising exercises with face validity 
to measure attainment of the objectives. National Assessment measures are 
criterion-referenced, and in fact National Assessment was in the business of 
develorji ig criterion-referenced measures long before the term became popular 
(Fniley, 1972) • While standardized tnsts compare each student with the average 
performance of all students, criterion-referenced measures indicate how well 
students as a group are achieving desirable goals. As National Assessment has 
demonstrated, criterion-referenced measures can be used in large-scale assessment 
programs and will provide information which norm-referenced, standardized tests 
have been unable to provide. To the extent that evaluotiop studies of social 
studies curricula rely heavily on standardized tests, evaluators— and their 
au( ences— are receiving a very limited picture of the impact of the programs 
being evaluated. 
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Two other features of National Assessment methodology also differ from stand- 
ardized testing programs. First, National Assessment exercises are prepared 
for the "high," the "average," and the "low" achieving student, while standard- 
ized tests usually enphas j the average range. As r\ result, sopietimes the 
best students "go off the top of the test" or the poorest "fall off the botto^• 
thus inadequately nieasurim either group (Finley, 1972). Second, since the 
goal of National Assessment is to assess the performance of groups of people, 
it is unnecessary to have ill the respondents answfr all the items for their 
age group. Instead National Assessment employs matrix sampling, in which each 
respondent in the ^.ssess'it; . sample takes only part (usually about one-twelfth) 
of the total pool of exercises. 

The materia ls of National ^^ssessment are also available to researchers. The 
objectives which were used as a basis for developing the first citizenship 
assessment are published and available from the U. S. Government Printing Office. 
These objectives were carefully revised and reviewed in 1959, as a basis for 
developing additional exerc es for the second citizenshio assessment. Having 
participated in the revision and reviewing of these objectives in 13^9, as did 
Jim Oswald, I can attest to c^-e painstaking efforts to include cr^l^ra^ aLtrioutt 
of citizenship attitudes and behavior in these objectives. Since these objectives 
are believed to reflect considerable consensus about good citizenship, a political 
scientist suggested that this presumed consensus could be tested by asking 
respondents whether they aj or disagree with clesc.iptions of civic behavior 
extracted from the objectives list. Comparisons between the two li:ts, developed 
about three years apart, reveal some shifts in civ^ic attitudes, even i? the 
choice of words. For exampi , the objective "help maintain law and order" 
became "support j^.st law and the rights of all individuals" in 1969. The 1969 
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''bjectives list is slated for publication soon. Eventually the citizenship 
'bjectives might be of interest in historical studies of trends in civic 
education in this country. 

'he exercises are also available to researchers. To date most of the "recyclinq" of 
itional Asses-nient exercises h:.3 been by state departments of education inter- 
Ut:d in conducting state assessments to obtain data paralleling the results 
f National Assessment. The state of Maine conducted an assessment of writing 
and citizenship last spring. They used virtually all of the available National 
Assessment citizenship exercises (i.e., those exercises which have been 
•ublicly released) which could be administered in paper-and-pencil format. 

he data obtained from the administration of the assessment exercises is, in 
i very real sense, at the disposal of social studies researchers. National 
Assessment data was not collected to prove any hypothesis or point of view, 
nor to pro^'ide scale scores on selected variables. To some people, these 
are considered limitations of the data and of the National Assessment prog^^am 

However, the mandate to the National Assessment, and to the American Institutes 
for Research as its cont^^acting agency for the citizenship assessment, was to 
provide information about accomplishment of important objectives "to teachers, 
administrators, school boards, curriculum developers, educational researchers, 
and other concerned citizens" in the hope that these people will bring National 
Assessment data to bear on their deci^,ions about how to improve education. 
Recommendations for action have been regarded as outside the scope of responsi- 
bility of National Assessment project staff. The original intent of National 
Assessnjent was to provide quantities of reliable information with the expecta- 
tion that this information v/ould be put to good use. But the very objectivity 
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of their stance has meant that the data have been little interpreted and seldom 
used. Perhaps the vast majority of educators are not prepared to use assessment 
data to improve edurntion in a systematic wav, but social studies r^'earrhor- 
can interpret, apply, and extend the find'*./ i\' '^ir^erou. ways, a. ^.U^ 
this paper and in the smaT group discussions to follow. 

The cri te>^ion-referenced nature of the data, with each exercise measuring at 
least one aspect of a single objective, permits comparisons with data from 
other sources. Individual exercises occasionally parallel items from other 
studies closely enough to permit comparisons of results, ":!iounh such narallel- 
ism was not deliberately sou-jht. Data from political soc^' ^ zation res-^arcn 
with children, the Purdue Studies of tc-.nagers' attitude:, ^nd the do.r^ on a f^v 
percent probability sample of high school students collected by Project TALENT 
are some examples of data sets which can be coordinated with the item results 
of each successive cycle of National Assessment, And with the 1969-70 citizen- 
ship assessment as a baseline, cornparisons can be made with results of subse- 
quent admistrations of the same and parallel items. Give'^i dozer jf :a banf ' . 
several with direct relevance for social studies research . anc the pr'^'*crit 
scarcity of funds to collect more data, researche-^ might seriously :ons^*^er 
putting others' data to new uses ( s s data ). 

At the inception of National Assessment citizenship and social studies lere 
designated as separate areas for assessment purooses. Whilt the logic of 
apparently "splitting off" citizenship from social studies has be^Mi question.' 
frequently, the effect has been co feus r^ore research attention on citizen- 
ship than it has typically received. The first cvcle of ohiective s ^.nJ e.--^ 
cises v/ere developed under contract by Educationa' Testing Service. The 



exercises were administered during the past year, and reports of results should 
be available during 1973. 
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Summary of Ob jec fives 
for Nafional Assessmer' of Citizenship Achieve, lenf 



SHOW CONCERN FOR THE WELL-BEING AND DIGNITY OF OTHERS. 

A. Treaf others wifh respect. 

B. Consider the consequences for others of their own actions. 
C Guard safety and health of others. 

D. Offer help to others in need. 

E. Support equal opportunity in educotion, housing, employment, and recreation, 

F. Are loyal to country, to friends, and to other groups whose values rhey 
share . 

G. Are ethical and dependable in work, sc':ool , and social situations. 

SUPPORT JUST LAW AND THE RIGHTS OF ALL INDIVIDUALS. 

A. Understand the need for law. 

B. Recognize specific constitutional rights and liberties. 
C: Defend rights and liberties of all kinds of people. 

D. Encourage ethical and lawful behavior in others. 

E. Comply with public laws. 

F. Oppose unjust rules, laws and authority by lawful means. 

KNOW THE MAIN STRUCTURE AND FUNCTIONS OF OUR GOVERNMENTS. 

A. Recognize basic governmental purposes. 

B. Understand the organization of federal and state governments. 

C. Know the political structure of their local community. 

D. Recognize the relationships of different levels of government. 

E. Recognize the importance of political opposition and interest groups. 

F. Recognize that democracy depends on the alertness and involvement of its 
citizens, and know how citizens can affect government. 

G. Know structure of school and student government. 

PARTICIPATE IN DEMOCRATIC CIVIC IMPROVEMENT. 

A. Believe that each person's civic behavior is important, and convey this 
belief to others. 

B. Favor organized civic action where it is needed. 

C. Activel; work for civic improvement. 

D. Participate in local, state and rational governmental processes. 

E. Apply democratic procedures effectively in small groups. 
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V UNDERSTAND IMPORTANT WORLD, NATIONAL AND LOCAL CIVIC PROBLEMS. 

A. Undersfand social canflicf among individuals, groups and nations and fhe 
difficulties in achieving peace and social harmony. 

B. Recognize haw different civic policies may affect people's efforts to meet 
their economic needs. 

C. Recognize major environmental problems and are aware of alternative civic 
solutions. 

D. See relations among civic problems and particular events. 

E. Can generate good ideas about causes and solutions for civic problems. 

VL APPROACH CIVIC DECISIONS RATIONALLY. 

A. Seek relevant information and alternative viewpoints on civicly important 
decisions. 

B. Evaluate civic communications and actions carefully as a basis for forming 
and changing their own views. 

C. Plan and organize civic tasks effectively., 

D. Support oper^, honest communication and universal education. 

Vll., HELP AND RESPECT THEIR OWN FAMILIES. 

A. Cooperate in home responsibilities and help provide for other family members. 

B. Instill civic values and skills in other family members. 



For further information contact 



Vincent Campbell ar Karen Fox 
American Institutes for Research 
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National Assessment 

of Educational Progress 
300 Lincoln Tower 
1860 Lincoln Streef 
Denver, Colorado 80203 
(303) 893-5200 



On fhe following pages are samples of exercises used in fhe first Cifizenship 
assessmenf. The percentages shown are results for the nation as a whole. 
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13:12-19 EXERCISE A3* - Object ives: Understand and oppose unequal 

17:13- 9 opj^ortunity in recreation. Help other individuals voluntarily. 

A: 4- 5 Recognize ways for citizens to influence government action. 

Ages 13, 17, and Adult: Interview 

A. Suppose you and some friends were walking by a public park. As you went 
by, some children of a minority group were stopped from entering the pork 
by a man at the gate who told them, "The pork is not for kids like you." 
Would you feel that you should do something about it? (Yes, No) 

(If respondent does not understand "minority group," explain that a minority 
group is a port of the population differin from others, as in race or reli- 
gion.) 

B. What could you do about it if you wanted to? 

(If respondent pauses before four responses are given, ask "Is there anything 
else you could do?" Stop after four responses OR when fh^ respondent 
answers "No" OR when no response is given in ten seconds.) 

Acceptable answers to B: Tell parents; report it ^ playground or related 
authority (city council, mayor, etc.); tell person doing it he was wrong; tell 
principal or teacher; write letters to newspaper; report it to police; take some 
form of social action (picket, handbills, signs, boycott); legal action; con- 
tact NAACP, ACLU or other civil liberti es groups. 

Unacceptable answers to B: Sneak them in bock c^ite; hit the man; talk to 
the children to see if they would behave; do nothing, take them to another 
park. 

Results 

Age 

J3 J7 Adult 

Felt they should act to help stop discrimination in 

a public oark (Yes to A) 82% 90% 79% 



Not administered to the in-school sample in one large western state, one 
southeastern county and one southwestern cil"y at the request of state or local 
authorities. 



EXERCISE A3 (continued) 



Age 

13 17 Adult 



Stated actions they could take to help stop public 
park discrimination (acceptoble answer to B) 



Stated 1 


or 


more 


actions they could take 


81% 


92% 


82% 


... 2 


or 


more 




39 


52 


50 


• • • 3 


or 


more 


• • • 


11 


17 


19 


. • • 4 


or 


more 




2 


3 


6 



Additional Informction 

Of those who indicated that they felt they should act to help stop discrimination 
in this situation, a high percentage also stated one or more things that they 
could do about it: 13-year olds (87%), 17-year olds (96%), Adults (93%). 
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